1. Field
The field of the invention relates generally to data compression and more particularly relates to a system and method for compressing financial data using data compression hardware.
2. Brief Discussion of Related Art
Lossless data compression methods in streaming database systems reduce storage requirements, improve access performance to stored data, and minimize use of computational resources to perform data compression and restoration. An often-unstated but intrinsically assumed goal for such streaming database systems is to provide uninterrupted access to compressed data. The conflicting nature of these goals, in a practical implementation of a streaming database system, generally results in compromised solutions that achieve gains toward one goal at the expense of another. Storage requirements may be reduced by transformations on the data, such as ordering data in columns or implementing record- or field-level data compression, but the cost is usually reduced performance for data access and increased computational requirements to perform the transformations.
Since lossless data compression is a computationally expensive operation, software compression solutions are not practical for high-performance database systems and are only adequate for database systems that do not have stringent performance requirements. Hardware accelerated data compression is one practical solution suitable for performance-hungry database systems. However, data compression hardware, as in any hardware resources, is subject to malfunction and requires a fail-safe mechanism to guarantee the integrity of and access to compressed data in the event of partial or total hardware malfunction.
Streaming database systems require random access to data, whether compressed or uncompressed. Any attempt to retrofit compression into an existing database, or to design compression into a newly constructed database, must provide a mechanism guaranteeing efficient random access to compressed data. Moreover, since not all data is compressible, both uncompressed and compressed data coexist in the database and the data access mechanism must be efficient for both types of data.